Preliminary Evaluation of Schema Matching Systems

While there have been some evaluations and surveys of these evaluations, the overall effectiveness of currently available automatic schema matching systems is largely unclear. This is mainly because either the evaluations were conducted in diverse ways making it difficult to assess the effectiveness of each single system, or they were based on previously published information rather than on actual evaluation. We performed a new evaluation of the state-of-the-art schema matching approaches based on comprehensive testing modern systems on various real-world schemas. The results obtained show that there is still no one matcher that performs best on all types of schemas used. Quality of mappings depends significantly on the approaches to schema matching used in various prototypes and on the tuning of the matcher to the particular schema type. The approach proposed in COMA to combine results of different schema matchers proved its effectiveness in our evaluation.